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DETAILED ACTION 

1 . The response received on August 20, 2004 has been placed in the file and was considaed by the examine-. 
An action on the merits follows. 

Response to Arguments 

2. The applicant's arguments, filed on August 20, 2004, have been fully considered. A response to these 
arguments is provided below. 

Drawing and Specification Objections 

Summary of Argument: Applicant has amended the specification and provided replacement drawing sheets 
to correct the previously objected to informalities. 

Examiner 's Response: The examiner appreciates the applicant's efforts to resolve these issues. The 
objections from the previous action are hereby withdrawn. However, a new specification objection will be provided 
below. 

37 CFR 1.75 Claim Objections 

Summary of Argument: Applicant has amended claim 19 to correct an improper claim dependency. 
Examiner 's Response: The previous objections undo - 37 CFR 1.75 are hereby withdrawn. The examiner 
appreciates the applicant's efforts to resolve this issue. 

Prior Art Rejections 

Summary of Argument: Applicant traverses the rejection in the prior office action of claims 1,2 4-9, and 
14-19 undo- 35 USC 102e as being anticipated by Szeliski et al (USPN 6,044,181). Specifically, applicant argues 
that Szeliski fails to disclose that the images being merged together to form a panoramic image are not e range 
images' as required by the claim. 

Examiner 's Response: Applicant's arguments have been fully considered and are persuasive. It appears as 
if the examiner misinterpreted the Szeliski reference by asserting that the overlapped images were range images. 
After taking a close look at the Szeliski reference, the examiner agrees that these images do not qualify as range 
images. 

The previous claim rejections are therefore withdrawn, but the applicant's arguments regarding the prior art 
rejections from the previous action have been rendered moot by the new grounds of rejection provided below. 



Application/Control Number: 09/803,802 
Art Unit: 2621 



Page 3 



Specification 

3. The disclosure is objected to because of the following informalities: 

The examiner requests that the applicant update the status of any US patent applications reference 
throughout the specification (e.g. 09/686,610 is now USPN 6,677,982). 

Also, the applicant makes reference to USPN 4,953,616 in paragraph [0005] of the specification. This is 
obviously a typo, and should be changed to 4,935,616. 

Also, the examiner suggests amending paragraph [0041] of the specification in order to more clearly 
identify the numbered elements. One non-exhaustive illustrative example is the sentence "A warping function and 
registration point is computed 110 and used to determine the differences in constant offsets of the relative 3D range 
values between image bundles from adjacent positions 1 12 ." This sentence is misleading because it appears that the 
'112' identifier is referring to 'adjacent positions. ' The prior paragraph in conjunction with Figure 1 clearly show 
that identifier 4 1 12' refers to the step of 'determining relative range differences. ' Thus, the language of paragraph 
[0041] is confusing and correction (this type of error occurs more than once in paragraph [0041]). Any other errors 
of this type that may exist in other paragraphs should be corrected as well. 

Appropriate correction is required. 

Claim Rejections - 35 USC §112 

4. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

5. Claim 8 is rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for failing to particularly 
point out and distinctly claim the subject matter which applicant regards as the invention. 

Referring to claim 8, the metes and bounds of the term "elsewhere" are not clearly defined. 

Claim Rejections - 35 USC §102 

6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the 
rejections undo* this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

7. Claims 1 and 16 are rejected under 35 U.S.C. 102(b) as being anticipated by R.T. Whitaker, J. Gregor, P.F. 
w~ Chen, University of Tennessee (hereinafter 'renowned institution') with the paper, "Indoor Scene Reconstruction 

from Sets of Noisy Range Images". This paper was published in 1999 at IEEE's Second International Conference 
on 3-D Imaging and Modeling. 
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With regard to claim 1, 'renowned institution' discloses a method for deriving a 3-D panorama from a 
plurality of images of a scene generated by a range imaging camera that produces ambiguities in range information 
(see the first paragraph of the 'Introduction': The reference describes deriving a 3-D model of an 'entire scene' (i.e. 
a panorama). The reference further describes that the images are 'laser range images' or 'LADAR data'. The 
applicant's dislcosure (see paragraph [0027], inter alia) explains that the claimed range imaging camera that 
produces the range information is a laser radar. Thus, the reference clearly anticipates this limitation. 

The 'renowned institution' further discloses acquiring a plurality of adjacent, overlapping range images 
(see Figure 4a-b). 

The 'renowned institution' further discloses providing offset data for the range images (pg. 1 it. col, final 
paragraph: The reference describes determining 'distances between views'. This is analogous to providing offset 
data as recited in the claim.) in order to recover corrected relative scene spatial information (pg. 1 right column, 
lines 4-7: The reference describes registering separate viewpoints and then integrating those viewpoints. This is 
analogous to recovering the relative scene spatial information. Furthermore, the great institution describes, in the 
second paragraph of the right column of page 2 of the reference, that a smoothing preprocessing operation is 
performed before all of the other processes. Smoothing qualifies as a type of 'correction'; therefore the reference 
also meets the claimed limitation of 'corrected relative scene spatial information.'). 

The 'renowned institution' further discloses detecting a relative range difference between adjacent range 
images as a constant offset (On page 1, right column, final paragraph, the refernce describes determining a distance 
between views. The reference further elaborates on the view registration (or pose determination, see pg. 1, right 
column, the first two lines of the second paragraph) method and the plane offsets (see page 4 - rt. column - the 
paragraph between equations 9 and 10) by defining variables coi and co 2 , which are plane offsets. This clearly shows 
that we have a constant offset between adjacenet range images as required by the claim. 

The 'renowned institution' further discloses using this offset data to correct for ambiguities in the relative 
ranges of the range images, thereby providing corrected range images. As was stated in the paragraph above, the 
'renowned institution' discloses determining range differences between adjacent range images. These offsets are 
analogous to the ambiguities recited in the claim (also see page 2 - left column - final paragraph: this passage 
makes explicit mention of the ambiguities associated with the range images). The passage cited in the above 
paragraph shows how the range differences are used in the determination of a translation which aligns, registers, or 
fits these images together, thereby providing corrected range images. 

The 'renowned institution' further discloses deriving a 3D panorama from the corrected range images. 
This was discussed above with respect to the preamble. As was already stated, the 'entire scene' derived by the 
'renowned institution' s's reference, is analogous to the claimed 'panorama'. These two terms are definitionally 
equivalent (see websters dictionary). 

With regard to claim 16, which merely adds a computer program for performing the steps of the method, 
this additional limitation is inherent in the 'renowned institution's disclosure. 
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Claim Rejections - 35 USC §103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set 

forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

9. Claims 2, 4, 5, 7, 8 and 14-18 are rejected undo - 35 U.S.C. 103(a) as being unpataitable over the 
combinatiion of e the 'renowned institution" and Hsieh et al. (USPN 6,011,558). The arguments as to the relevance 
of the 'renowned institution' reference as applied above are incorporated herein and will not be repeated. 

Referring to claim 2, the 'renowned institution' discloses that the scene spatial information is provided as 
image values in a local 3D coordinate system of each of the images (pg 2 - rt. column - 1 st paragraph), the 
'renowned institution' further discloses performing transformation calcuations on a 3D grid (see pg. 7 - rt column - 
1 st paragraph). The examiner feels that a reasonably compelling argument could be made that the transformation to 
this 3D grid from the local 3D scanning coordinates is inherent in the system. However, the examiner agrees that 
this feature is not explicitly stated in the reference, and therefore will err on the side of caution and meet the 
limitation more convincingly with the teachings of Hsieh. 

The Hsieh reference discloses transforming the image values from the local coordinate system of each 
image to a world coordinate system, thereby providing a transformed image (Hsieh col. 1 line 43 - col. 2 line 9 in 
conjunction with Figure 2: The reference discloses transforming the captured image plane's 10 coordinate system 
P(x,y) to the cylindrical map coordinates p(u,v). The cylindrical map coordinates are analogous to the world 
coordinate system recited in the claim. This coordinate transformation is performed for all of the captured image 
planes. Thus, the reference meets the plural claim language. 

Hsieh further discloses warping a plurality of transformed images onto a cylindrical surface (Hsieh col. 2 
lines 7-10); registering adjacent warped images (Hsieh col. 4 lines 33-35 in conjunction with figures 4a-b); and 
deriving the panorama using the warped images (Hsieh col. 2 lines 16-17). 

It would have been obvious to one reasonably skilled in the art at the time of the invention to modify 'the 
renowned institutions' method of deriving a 3D panorama by transforming and mapping the captured image planes 
into a cylindrical shape as taught by Hsieh. Such a modification would have allowed for the building of a virtual 
environment map (Hsieh col. 1 lines 40-41). 

Referring to claim 4, Hsieh further discloses that the images are acquired by horizontally rotating a camera 
about its optical center (Hsieh col. 1 lines 17-20). The only type of axis that a earner can be 'horizontally rotated 
about' is a vertical axis. Thus, the additional limitation of claim 4 is met by Hsieh. With regard to the claim 4 
limitation of (X, Y,Z) coordinates and/or images, this limitation has already been addressed with respect to the claim 
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2 limitation of a 3D coordinate and/or image. It is well known that (X, Y,Z) coordinates are merely a synonym for 
3D coordinates. All of the 3D limitations were met by the 'renowned institution' reference. 

Referring to claim 5, the limitations were addressed in the above claim 1 rejection. 

Referring to claim 7: the world 3D coordinate system set forth in the prior art qualifies as an arbitrary 3D 
coordinate system. In fact, it is difficult to imagine a coordinate system that didn't qualify as 'arbitrary'. An 
arbitrary coordinate system is one with no limitations (see websters dictionary definition). Thus, by definition, it 
appears as if any 3D coordinate system would limitations of the claim. Furthermore, a 3D coordinate system is 
merely a collection of unit vectors used to determine the position of points in space. Since coordinate systems are an 
independent entity which establish a frame of reference, they themselves are not measured according to any outside 
frame of reference. Therefore, it could be said that all coordinate systems are arbitrary, since they are limited only 
to the discretion of an individual. As a result, we can conclude that the combination of the 'renowned institution' 
reference and the Hsieh reference meet the claimed limitations. 

Referring to claim 8: this claim fails to further limit its parent claim. Any 3D world coordinate system will 
either be selected from the local 3D coordinate system, or a 3D coordinate system defined elsewhere. These are the 
only two possible options. As a result, the 3D coordinate system set forth in the prior art meets the limitations of the 
claim. 

With regard to claims 14 and 15, which merely call for a system for performing the method of claims 1 and 
4, Hsieh discloses such a system as can be seen in Fig. 6. 

With regard to claims 16, 17, 18, which merely adds a computer program for pa-forming the steps of the 
method of claims 1, 4, and 5. This additional limitation is inherent in the combination of the 'renowned institution' 
reference and Hsieh. 

10. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over the combination of 'renowned 
institution' and Hsie, and further in view of Ray (USPN 6,023,588). The arguments as to the relevance of the 
aforesaid combination as applied above are incorporated herein. 

Referring to claim 6, the aforesaid combination discloses a range imaging camera for generating range 
images. The combination also discloses producing a 3D panorama. The combination fails to expressly disclose 
that the range images and the 3D panorama are in color. Ray, however, in the same field of endeavour, discloses 
using color images to produce a color 3D panorama (Ray col. 6 lines 16-20 and col. 8 line 1 1). It would have been 
obvious to one reasonably skilled in the art at the time of the invention to modify the combination of 'renowned 
resolution' and Hsieh' s method for producing a panoramic by merging range images by adding the element of 
color to the images as taught by Ray. Such a modification would have allowed for a more aesthetically pleasing 
and visually descriptive result. 

1 1. Claims 9 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable over the combination of 
'renowned institution' and Hsieh as applied to claim 4 above, and further in view of Lipscomb et al. (USPN 
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5,796,386). The arguments as to the relevance of the aforesaid combination as applied above are incorporated 
herein. 

Referring to claim 9, the aforesaid combination discloses transforming the corrected relative spatial 
information, but fails to expressly disclose tha the transformation comprises forming a homogenous transformation 
matrix. Lipscomb, however, discloses a transformation between coordinate systems which comprises forming a 
homogenous transformation matrix (Lipscomb col. 5 lines 56-63). It would have been obvious to one reasonably 
skilled in the art at the time of the invention to modify Hsieh's coordinate transformation by utilizing a homogenous 
transformation matrix as taught by Lipscomb. Such a modification would have allowed for a computational friendly 
method of transforming between two coordinates, which could be easily programmed and executed on a computer. 

12. Claims 10-1 1 are rejected under 35 U.S.C. 103(a) as being obvious over the combination of 'renowned 
institution' and Hsieh as applied to claim 4 above and further in view of Ray et al. (U.S. Patent No. 6,1 18,946 Bl). 
The arguments as to the relevance of Szeliski et al. in the rejection of claim 4 above are incorporated herein. 

Szeliski et al. does not disclose that each image is captured as a bundle of associated images, nor does the 
reference disclose the particulars of such a bundle. 

Ray et al., in the same field of endeavor of image processing and the same problem solving area of range 
image formation discloses capturing an image in such a manna* and the particulars of such image capture. 

In particular, as applied to claim 10, Ray et al. discloses that each image is captured as a bundle of 
associated images, said bundle including a plurality of phase images each incorporating the effect of a 
predetermined modulation frequency together with a phase offset unique for each image (see Fig. 3 and column 4, 
lines 38-55: The reference describes that each image is captured as a bundle of associated images and that the bundle 
includes at least three phase images each incorporating the effect of a predetermined modulating frequency and a 
phase offset.). 

As applied to claim 11, Ray et al. discloses that each range image is generated from a respective plurality of 
phase images associated with each bundle (see column 4, line 42: The reference describes that the range image is 
generated from at least three phase images.). 

It would have been obvious to one reasonably skilled in the art at the time of the invention the range image 
capture system described in the combination of 'renowned institution' and Hsieh by capturing a bundle of images to 
produce the range image as taught by ray. Such a modification would have allowed for a resulting image with 
increased spatial resolution (ray col. 1 49-51). 

13. Claims 12 and 13 are rejected under 35 USC 103(a) as being obvious over the combination as applied to 
claim 10 above, and further in view of Ray (USPN 6,456,793). The arguments as to the relevance of the aforesaid 
combination as applied above are incorporated herein. 
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As applied to claim 13, which is representative of claim 12, Ray et al. discloses that the bundle also 
includes a color image (see column 7, lines 46-48: The reference describes that the bundle includes a range image.). 

It would have been obvious to a person of ordinary skill in the art at the time the invention was made to 
modify Szeliski et al. by capturing each image as a bundle of associated images as taught by Ray et al. because such 
a system allows for the "capture [of] ranging information without the sacrificing color information that would 
otherwise be available for capture" (see Ray et al.: column 4, lines 11-13). 

Therefore, it would have been obvious to combine Szeliski et al. with Ray et al. to obtain the invention as 
specified in claims 12-13. 

Conclusion 

14. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure, 
o Hall-Holt et al. (USPN 6,754,370) describes range scanning a moving scene 

o Koffman et al. (USPN 6,542,249) describes many processing operations performed in the 3D range image 
environment. 

15. Any inquiry concerning this communication or earlier communications from the examiner should be 
directed to Patrick L Edwards whose telephone number is (703) 305-6301. The examiner can normally be reached 
on 8:30am - 5:00pm M-F. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Leo Boudreau 
can be reached on (703) 305-4706. The fax phone number for the organization where this application or proceeding 
is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent Application Information 
Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR 
or Public PAI& Status information for unpublished applications is available through Private PAIR only. For more 
information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the 



Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 




